Search for: All records

Recently, researchers have initiated a new wave of convergent research in which Mixed Reality visualizations enable new modalities of human-robot communication, including Mixed Reality Deictic Gestures (MRDGs) – the use of visualizations like virtual arms or arrows to serve the same purpose as traditional physical deictic gestures. But while researchers have demonstrated a variety of benefits to these gestures, it is unclear whether the success of these gestures depends on a user’s level and type of cognitive load. We explore this question through an experiment grounded in rich theories of cognitive resources, attention, and multi-tasking, with significant inspiration drawn from Multiple Resource Theory. Our results suggest that MRDGs provide task-oriented benefits regardless of cognitive load, but only when paired with complex language. These results suggest that designers can pair rich referring expressions with MRDGs without fear of cognitively overloading their users. 

Abstract The Institute for Student‐AI Teaming (iSAT) addresses the foundational question:how to promote deep conceptual learning via rich socio‐collaborative learning experiences for all students?—a question that is ripe for AI‐based facilitation and has the potential to transform classrooms. We advance research in speech, computer vision, human‐agent teaming, computer‐supported collaborative learning, expansive co‐design, and the science of broadening participation to design and study next generation AI technologies (called AI Partners) embedded in student collaborative learning teams in coordination with teachers. Our institute ascribes to theoretical perspectives that aim to create a normative environment of widespread engagement through responsible design of technology, curriculum, and pedagogy in partnership with K–12 educators, racially diverse students, parents, and other community members. 

Classroom orchestration requires teachers to concurrently manage multiple activities across multiple social levels (individual, group, and class) and under various constraints. Real-time dashboards can support teachers; however, designing actionable dashboards is a huge challenge. This paper describes a participatory design study to identify and inform critical features of a dashboard for displaying relevant, actionable, real-time data. We leveraged a Sense-Assess-Act framework to present dashboard mockups to teachers for feedback. Although the participating teachers differed in how they would use the presented information (during class or after class as a post hoc analysis tool), two common emerging themes were that they wanted to use the data to a) better support their students and b) to make broader instructional decisions. We present data from our study and propose a customizable, mobile dashboard, that can be adapted to a teacher's specific needs at a specific time, to help them better facilitate learning activities.